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The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )£3 Responsive to communication(s) filed on 30 August 2000 . 
2a)Q This action is FINAL. 2b)^ This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-51 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) IEI Claim(s) 29-45 is/are allowed. 

6) E3 Claim(s) 1-5,10.19-28 and 46-51 is/are rejected. 

7) K Claim(s) 6-9 and 11-18 is/are objected to. 

^ 8)D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9)D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

1 1) D The proposed drawing correction filed on is: a)D approved b)Q disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)DAII b)D Some*c)D None of: 

1 -D Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attach ment(s) 

1) IS Notice of References Cited (PTO-892) 4) Q Interview Summary (PTO-413) Paper No(s). . 

2) £3 Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) O Notice of Informal Patent Application (PTO-152) 

3) K Information Disclosure Statement(s) (PTO-1449) Paper No(s) 3 . 6) Q Other: 
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DETAILED ACTION 



Objection to Color Drawings 



The application has been filed with informal drawings which are acceptable for 



examination purposes only. Formal drawings will be required when the application is allowed. 

2. The drawings are considered to be informal because they fail to comply with 37 CFR 
1.84(a)(1) which requires black and white drawings using India ink or its equivalent. 

3. Photographs and color drawings are acceptable only for examination purposes unless a 
petition filed under 37 CFR 1.84(a)(2) or (b)(1) is granted permitting their use as formal 
drawings. In the event applicant wishes to use the drawings currently on file as formal drawings, 
a petition must be filed for acceptance of the photographs or color drawings as formal drawings, 
Any such petition must be accompanied by the appropriate fee as set forth in 37 CFR 1.1 7(i), 
three sets of drawings or photographs, as appropriate, and, if filed under the provisions of 37 
CFR 1.84(a)(2), an amendment to the first paragraph of the brief description of the drawings 
section of the specification which states: 

4. The file of this patent contains at least one drawing executed in color. Copies of this 
patent with color drawing(s) will be provided by the Patent & Trademark Office upon request 
and payment of the necessary fee. 

5. Color photographs will be accepted if the conditions for accepting color drawings has 
been satisfied. 
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Allowed and Allowable Subject Matter 

6. Claims 29-45 are allowed, and claims 6-9 and 1 1-18 are objected to as being dependent 
upon a rejected base claim, but would be allowable if rewritten in independent form including all 
of the limitations of the base claim and any intervening claims. 

7. The following is a statement of reasons for the indication of allowable subject matter: No 
prior art anticipates or suggests, in a facial expression transformation system, a second code book 
in addition to a first code book and a training system of expressions (claim 29). Nor does any 
prior art anticipate or suggest, in a facial expression transformation method, computing a set of 
linear predictors aj, one for each coordinate of ga, given a set of n expression vectors for a face to 
be transformed, g a i... n > and a corresponding set of vectors for a target face, gbi... n , by solving 3m 
linear least square systems of the form aj • g a i = gbiUy=l..«, wherein said computing comprises 
using only a subset of points for each g aj . (claim 6), controlling the spread of singular values 
when computing a pseudoinverse to solve for the aj (claim 8), projecting a pattern onto a face for 
the purpose of ascertaining structure data (claim 11), capturing both specularly-suppressed 
reflection data and structure data from the simultaneous illumination of a second person's face 
with one light source that is polarized and one structured light source that projects a pattern onto 
the face (claim 12), and processing surface normals and image data to derive an albedo map in 
the process of providing data defining a second set of facial expressions (claim 16). Claims 30- 
45 are allowed, and claims 7, 9, 13-15 and 17-18 are allowable, because they depend on claims 
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29, 6, 8, 12 and 16, respectively. 



Claim Rejections - 35 USC§102 



8. The following is a quotation of 35 U.S.C. 102(e) which forms the basis for all novelty- 
related rejections set forth in this Office action: 

A person shall be entitled to a patent unless — 

(e) the invention was described in a patent granted on an application for patent by another filed in the United States 
before the invention thereof by the applicant for patent, or on an international application by another who has 
fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before the invention 
thereof by applicant for patent. 

9. Claims 19-28 are rejected under 35 U.S.C. 102(e) as being anticipated by LaChapelle 
(U.S. Pat. No. 6,163,322). 

10. LaChapelle, in disclosing a method and apparatus for providing a real-time animation 
utilizing a database of postures, also discloses, with respect to claim 19, one or more computer- 
readable media having computer-readable instructions thereon which, when executed by a 
computer, cause the computer to: 

• operate on a training set of expressions from one person (database of expressions; col.8, 
11.22-25) and corresponding expressions from a code book of another person (col.4, 11.29- 
46) to compute a linear transformation function from the training set and their 
corresponding expressions (mapping template; col. 9, 11.43-51. The neutral facial 
expression Eo is the source of the facial expressions in the database; see col.8, 11.45-48); 
and apply the transformation function to a plurality of expressions from the code book to 
provide a synthetic set of expressions (col. 13, 11.49-65). 
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1 1 . Concerning claim 20, LaChapelle further discloses the instructions causing the computer 
to use the synthetic set of expressions to transform expressions from the one person into 
expressions of the other person (col.9, 11.43-51). 

12. Regarding claim 21, LaChapelle further discloses instructions causing the computer to 
transform expressions from the one person that are different from those expressions comprising 
the code book expressions (mapping template, col.9, 11.43-51). 

13. With respect to claim 22, LaChapelle further discloses instructions causing the computer 
to transform expressions from the one person that are different from those expressions 
comprising the code book expressions (markers, col.9, 11.43-47). 

14. Regarding claim 23, LaChapelle further discloses instructions causing the computer to 
transform facial expressions (mapping template, col.9, 11.43-51). 

15. Concerning claim 24, LaChapelle further discloses a facial expression transformation 
system comprising: 

• a code book embodied on a computer-readable medium, the code book containing data 
defining a first set of facial expressions of a first person (col.4, 11.29-46); 

• data embodied on a computer-readable medium, the data defining a second set of facial 
expressions, the second set of facial expressions providing a training set of expressions 
of a second person who is different from the first person (database of expressions; col. 8, 
11.22-25); and 
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• a transformation processor configured to derive a transformation function from the 
training set of expressions and corresponding expressions from the first set of 
expressions (mapping template, col. 9, 11.43-51. The neutral facial expression Eo is the 
source of the facial expressions in the database; see col. 8, 11.45-48). 

16. Regarding claim 25, LaChapelle further discloses the transformation processor 
comprising a linear transformation processor (col. 10, 11.34-38). 

17. With respect to claim 26, LaChapelle further discloses the expression transformation 
system comprising a synthetic set of expressions embodied on a computer-readable medium, the 
synthetic set of expressions being derived by applying the transformation function to the code 
book expressions (col. 9, 11.43-51). 

18. Concerning claim 27, LaChapelle further discloses the transformation function 
compensating for differences in the size and shape of the faces of the first and second persons 
(col.10, 11.54-57). 

19. Finally, regarding claim 28, LaChapelle further discloses the transformation processor 
deriving the transformation function by: 

• representing each expression as a 3wz-vector that contains x, y, z displacements at m 

standard sample positions ( i5 col.4, 11.63-66); and 

• computing a set of linear predictors aj, one for each coordinate of ga, given a set of n 

expression vectors for a face to be transformed, gai... n , and a corresponding set of vectors 
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for a target face, gbi . n, by solving 3m linear least squares systems of the form: aj * gaj = 
gbi[j]>* = l".« (computation of in col. 5, 11.8-18). 

20. Therefore, in view of the foregoing, claims 19-28 are rejected as being anticipated under 
35 U.S.C. 102 by LaChapelle. 

Claim Rejections - 35 USC §103 

21 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth 
in section 102 of this title, if the differences between the subject matter sought to be patented and the 
prior art are such that the subject matter as a whole would have been obvious at the time the invention 
was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

22. Claims 1-5 are rejected under 35 U.S.C. 103(a) as being unpatentable over LaChapelle in 

view of Georgiev (U.S. Pat. No. 6,351,269). 

23. LaChapelle discloses, with respect to claim 1, a facial expression transformation method 
comprising: 

• defining a code book containing data defining a first set of facial expressions of a first 
person (col.4, 11.29-46); 

• providing data defining a second set of facial expressions, the second set of facial 
expressions providing a training set of expressions of a second person who is different 
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from the first person (database of expressions; col.8, 11.22-25); and 
• deriving a transformation function the training set of expressions and corresponding 
expressions from the first set of expressions (mapping template, col. 9, 11.43-51). 

24. However, LaChapelle does not disclose applying the transformation function to the first 
set of expressions to provide a synthetic set of expressions. This element is disclosed by the 
Georgiev method of multiple image morphing at col.4, 11.47-61. 

25. Therefore, it would have been obvious to one of ordinary skill in the art to incorporate the 
Georgiev morphing method into the LaChapelle animation method. This would provide 
flexibility in modifying images (Georgiev, col.4, 11.61-65). 

26. The other claims in this rejection will now be considered. Concerning claim 2, 
LaChapelle discloses the training set of expressions as containing fewer expressions than the 
code book (this is possible because creation of the number of expressions via tracking 
movements of the face of a performer does not depend on the number of expressions in the 
database. See col.4, 11.29-46.). 

27. Concerning claim 3, LaChapelle further discloses the transformation function 
compensating for differences in the size and shape of the faces of the first and second persons 
(col. 10, 11.54-57). 

28. Regarding claim 4, LaChapelle further discloses the derivation of the transformation 
function comprising a linear transformation processor (col. 10, 11.34-38). 
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29. Finally, regarding claim 5, LaChapelle further discloses the transformation processor 
deriving the transformation function by: 

• representing each expression as a 3/n-vector that contains x, y, z displacements at m 

standard sample positions ( i, col. 4, 11.63-66); and 

• computing a set of linear predictors aj, one for each coordinate of ga, given a set of n 

expression vectors for a face to be transformed, g a i„.n> and a corresponding set of vectors 
for a target face, gbi... n > by solving 3m linear least squares systems of the form: aj • gai = 
gbiDy = l— w (computation of in col. 5, 11.8-18). 

30. Therefore, in view of the foregoing, claims 1-5 are rejected as being unpatentable under 
35 U.S.C. 103 by LaChapelle and Georgiev. 

31. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over LaChapelle in 
view of Georgiev and further in view of Parke et al., Computer Facial Animation ("Parke"). 

32. Neither LaChapelle nor Georgiev disclose illuminating the second person's face with 
illumination and contemporaneously capturing structure data describing the face's structure and 
reflectance data describing reflectance properties of the face from the illumination. However, 
these elements are disclosed by the Parke animation textbook. Illuminating the second person's 
face is disclosed by Parke in Section 5.7.2, pp.183-185, and capturing the structure data is 
disclosed by Parke at Section 5.6, p. 170, first full paragraph. 
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33. Therefore, it would have been obvious to one of ordinary skill in the art at the time this 
invention was made to incorporate the illumination elements of Parke into the LaChapelle- 
Georgiev animation apparatus. This would provide optimal lighting for whatever purpose is 
needed (Parke, pp. 183- 185). 

34. Therefore, in view of the foregoing, claim 10 is rejected as being unpatentable under 35 
U.S.C. 103 by LaChapelle, Georgiev and Parke. 

35. Finally, claims 46-51 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Parke in view of Georgiev. 

36. Parke further discloses a method of animating facial features comprising defining a 
subdivision surface that approximates geometry of a plurality of different faces on p. 94 in the 
section marked "Fixed Topology". 

37. However, Parke does not disclose fitting the same subdivision surface to each of the 
plurality of faces to establish a correspondence between faces and using the correspondence 
between the faces to transform an expression of one face into an expression of another face. 
These elements are disclosed by Georgiev at col.4, 11.47-61. 

38. Therefore, it would have been obvious to one of ordinary skill in the art at the time this 
invention was made to incorporate the Georgiev method of establishing correspondence between 
faces into the Parke method of modeling. This would provide flexibility in modifying images 
(Georgiev, col.4, 11.61-65). 
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39. The other claims in this rejection will now be considered. With respect to claim 47, 
Parke discloses: 

• measuring 3-dimensional data for a plurality of different faces to provide corresponding 
face models (Section 3.4, p.66, first sentence of first paragraph and first bullet); 

• defining only one generic face model that is to be used to map to each corresponding 
face model (p.94, section marked "Fixed Topology"); and 

• selecting a plurality of points on the generic face model that are to be mapped directly to 
corresponding points on each of the corresponding face models (p.94, section marked 
"Fixed Topology", selection implied by second sentence). 

40. However, Parke does not disclose fitting the generic face model to each of the 
corresponding face models, said fitting comprising mapping each of the selected points directly 
to the corresponding points on each of the corresponding face models. This element is disclosed 
by Georgiev at col. 4, 11.47-61. 

41 . Therefore, it would have been obvious to one of ordinary skill in the art at the time this 
invention was made to incorporate the Georgiev method of fitting the generic face model to the 
corresponding face model into the Parke method of modeling. This would provide flexibility in 
modifying images (Georgiev, col. 4, 11.61-65). 

42. The other claims in this rejection will now be considered. Concerning claim 48, Parke 
discloses defining a subdivision surface from a base mesh structure, the subdivision surface 
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containing a plurality of vertices and approximating the geometry of the face models (p. 95, 
Figure 3.17 and first full paragraph); and manipulating only the positions of the vertices of the 
subdivision surface (p.94, section marked "Fixed Topology", next to the last sentence). 

43. Regarding claims 49 and 50, Parke discloses manipulating a base mesh that defines a 
subdivision surface without altering the connectivity of the base mesh (p.94, section marked 
"Fixed Topology", next to the last sentence). 

44. Finally, regarding claim 51, Parke discloses using a laser range scan to measure the 3- 
dimensional data at p. 78, Section 3.7, first two sentences. 

45. Therefore, in view of the foregoing, claims 46-5 1 are rejected as being unpatentable 
under 35 U.S.C. 103 by Parke and Georgiev. 



Any inquiry concerning this communication or earlier communications from the Office 
should be directed to the examiner, Lance Sealey, whose telephone number is (703) 305-0026. 
He can be reached from 7:00 am-3:30 pm Monday-Friday EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mark Zimmerman, can be reached at (703) 305-9798. 

Any response to this action should be mailed to: 



Conclusion 



Commissioner of Patents and Trademarks 
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Washington, D.C. 20231 

or faxed to: 

(703) 872-9314 (for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the Technology Center 2600 Customer Service Office at (703) 306-0377. 

Respectfully submitted, 
Lance W. Sealey, examiner 



